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ABSTRACT 



A method and apparatus for transcoding macroblocks of a 
high definition television signal to co-sited macroblocks of 
a standard definition television signal wherein SO macrob- 
locks are derived directly from co-sited HD macroblocks, 
i.e. without having to compute the SD macroblock informa- 
tion from its corresponding SD image sequence. 
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METHOD AND APPARATUS FOR TRANSCODING 
A DIGITALLY COMPRESSED HIGH DEFINITION 
TELEVISION BITSTREAM TO A STANDARD 
DEFINITION TELEVISION BITSTREAM 

BACKGROUND OF THE INVENTION 

[0001] The invention relates to techniques for manipulat- 
ing digitally compressed and coded data (for example video 
information) in order to convert it from one format (or 
specification) to another. Such a method will be referred to 
herein as digital transcoding, and a device with such a 
functionality will be referred to as a digital transcoder 

[0002] The digital video compression standard developed 
by International Standardization Organization's (ISO) Mov- 
ing Picture Expert Group (MPEG) is becoming a key 
technology in the delivery of digital video programs over a 
wide variety of media such as terrestrial broadcasting, 
telecommunication, and cable. It is now almost certain that 
a digital HDTV (high definition television) standard similar 
or compatible with standards recommended by MPEG will 
be used for terrestrial HDTV transmission in North America 
and Europe. Similar technology will also be used to provide 
HDTV and standard definition television (SDTV) over 
cable, phone, fiber optic, satellite and ISDN networks. 

[0003] U.S. Pat. No. 5,243,428 discusses the MPEG stan- 
dard as well as its block and frame coding protocols. This 
patent is incorporated by reference herein. Further details 
about inter/intra frame and MPEG like video coding can also 
be found in the following references which are also incor- 
porated by reference herein: 

[0004] MPEG: A Video Compression Standard For 
Multimedia Applications; Le Gall, Communications 
of the ACM, Vol.34, No. 4, April, 1991. 

[0005] Advanced Digital Communications^ Feher, 
Prentice-Hall Inc., Englewood Cliffs, NJ. (1987); 

[0006] The Digital Simulcast AD-HDTV Coding System, 
IEEE Transactions on Consumer Electronics, Vol. 38, No. 4, 
November, 1992; 

[0007] Irformation Technology-Generic Coding of 
Moving Pictures and Associated Audio , ISO/I EC 
committee draft, November 1993; and 

[0008] Test Model 5. Draft-Test Model Editing Com- 
mittee, ISO/IEC, April, 1993. 

[0009] SDTV is defined herein as a digitally encoded 
television signal which can deliver a television picture 
comparable in overall format and resolution to conventional 
(e.g. NTSC or PAL) type television pictures. Using tech- 
niques for coding HDTV source signals into conventional 
television channel bandwidths (e.g. 6 MHz.), several SDTV 
programs can be provided on each channel instead of a 
single HDTV program. 

[0010] Introduction of HDTV will probably begin before 
receivers capable of reproducing the full HDTV source 
signal are either available or affordable by most viewers. 
There will therefore be a need to convert HDTV signals to 
SDTV signals (i.e. transcode them) so that they can be 
further processed for display on conventional (e.g. NTSC) 
television receivers which will only be able to decode and 
display standard definition video. 



[0011] A digital transcoder may be located al an interme- 
diate stage in the transmitting chain or as part of a telecom- 
munication network such as al a head-end or at a network 
switch. As currently envisioned, a transcoder will receive the 
HDTV signals) from a central location via satellite or other 
network communications link and transcode one or more 
SDTV signals from respective HDTV signals. Both HDTV 
and SDTV signals will then be transmitted to the home. 

[0012] Although initially the transcoding equipment will 
be placed in the transmission chain requiring channels to be 
provided for both HDTV and SDTV signals, low cost ICs 
will eventuaUy become available to enable the transcoder to 
migrate to the consumer's home. The advantage of having 
the transcoder in receiving chain is that only the HDTV 
signal will actually have to be transmitted (rather than 
simulcast with the SDTV signal) and channels occupied by 
the SDTV signals will be utilized for other uses. 

[0013] Presently, transcoding from HDTV to SDTV is 
accomplished by completely decoding the HDTV signal to 
form a sequence of high definition images ("HD image 
sequence"), The HD image sequence must then be filtered 
and subsampled to extract a sequence of lower definition 
images ("SD image sequence"). The SD image sequence 
must then be processed to compute SD macroblock infor- 
mation, for example macroblock type information, motion 
vector information and quantizer information, in order to 
encode it. However, as is the case with most broadcast 
quality video systems, a complete encoder is expensive and 
it therefore would not be practical to include one in a cost 
effective transcoder designed to be used in the receiving (or 
transmitting) chain. 

[0014] An object of the instant invention is, therefore, to 
provide a method and apparatus for performing cost effec- 
tive transcoding which avoids having to compute SD mac- 
roblock information from the SD image sequence. 

SUMMARY OF THE INVENTION 

[0015] The instant invention provides a method and appa- 
ratus for 

[0016] decoding an HDTV signal to provide an HD 
image sequence and HD macroblock information 
pertaining to the coded macroblocks of the HDTV 
signal, for example picture type information, mac- 
roblock type information, motion vector information 
and quantizer information; 

[0017] filtering and subsampling the HD image 
sequence to provide an SD image sequence; and 

[0018] using the HD macroblock information to 
directly derive corresponding SD macroblock infor- 
mation (e.g. picture type information, macroblock 
type information, motion vector information and 
quantizer information) necessary for encoding co- 
sited SD macroblocks. 

[0019] By processing the HD macroblock information 
directly to derive the SD macroblock information, the inven- 
tion avoids the necessity of completely analysing the SD 
image sequence in order to derive the SD macroblock 
information. This simplifies the SD encoding process and 
apparatus since it requires much less memory and less 
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computational complexity than the prior art method, and 
therefore can be effectively implemented in the receiving 
chain. 

[0020] The preferred embodiments described in this appli- 
cation relate to transcoding a compressed HDTV signal to a 
compressed SDTV signal. In general, the same techniques 
can be applied to transcoding from any given higher reso- 
lution and bit-rate bitstream to a lower resolution and 
bit-rate bitstream. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0021] FIG. 1 is a block diagram of a preferred embodi- 
ment of a transcoder which implements the invention; 

[0022] FIG. 2 illustrates the relationship between co-sited 
HD macroblocks and SD macroblocks; and 

[0023] FIG. 3 illustrates the relationship between co-sited 
HD macroblock motion vectors and SD macroblock motion 
vectors. 

DETAILED DESCRIPTION OF PREFERRED 
EMBODIMENTS OF THE INVENTION 

[0024] The invention transcodes a compressed HD (high 
definition) video data bitstream into a compressed SD 
(lower, e.g. "standard" definition) data bitstream by utilizing 
HD macroblock information decoded from the HD bitstream 
to directly derive SD macroblock information. As used 
herein, "directly" means deriving SD macroblock informa- 
tion from HD macroblock information without having to 
cmpute the SD macroblock information from its correspond- 
ing SD image sequence. 

[0025] The references incorporated herein discuss, for 
example, the MPEG digital video protocol and encoders and 
decoders which can be used to provide both HD and SD 
digital signal processing. Details of the operation of digital 
compression and coding/decoding operations and equipment 
are therefore not treated in detail herein. 

[0026] The HD data bitstream provides coded information 
(for example coeflBcients, quantization scahng information 
and motion vectors) related to three types of coded picture 
types. They are I (intraframe) coded pictures, P (forward 
prediction interframe) coded pictures and B (bidirectional 
prediction interframe) coded pictures. Each P picture can 
include either forward predicted or intraframe coded indi- 
vidual macroblocks. Each B picture can include cither 
forward predicted, bidirectionally predicted or intraframe 
coded individual macroblocks. Each I picture can include I 
coded macroblocks only. The HD data bitstream is decoded 
to re-form the originally encoded HD macroblocks of pixels 
(for example 16x16 pixels). These macroblocks are then 
further processed to form an HD image sequence. As shown 
in FIG. 1, which describes a preferred embodiment of the 
transcoder, the HD information and HD image sequence are 
derived in HD decoder 10, The HD image sequence is then 
filtered and subsampled to form an SD image sequence in 
subsampler 20 which is then sent to partial SD encoder 40. 

[0027] Unlike the prior art, which further processes the SD 
image sequence in order to compute SD macroblock infor- 
mation, the instant invention directly derives SD macrob- 
lock information from HD macroblock information to 



encode the SD image sequence provided by subsampler 20 
into an SD coded data bitstream in partial SD encoder 40. 

[0028] HD macroblock information comprises mode 
information for each group of four "co-sited" HD macrob- 
locks and provides the mode information to mode selection 
processor 50. The mode information includes: 

[0029] a) the type of prediction used for each of the 
macroblocks of the group, which includes forward 
predicted, bidirectional predicted or intraframe 
coded i.e. no prediction (intra), and whether a field or 
frame prediction is used; 

[0030] b) whether or not each respective macroblock 
comprises quantizer scale information; 

[0031] c) whether or not each respective macroblock 
comprises residual coeflBcienl data and if so whether 
it is field or frame DCT coded; and 

[0032] d) whether or not each respective macroblock 
comprises motion information. 

[0033] Co -sited macroblocks are defined herein as the 
group of HD macroblocks forming the portion of an HD 
picture from which a corresponding SD macroblock of the 
SD picture is to be formed. The relationship between co- 
sited HD macroblocks and SD macroblocks is shown in 
FIG. 2. 

[0034] The relationship between each SD macroblock 
located at the position x,y within the SD picture, to a 
corresponding portion of the HD picture (of size Sx, Sy) 
located at position X,Y within the HD picture is expressed 
by the equations X=R*x, Y^R*y, and Sx=Sy=16*R, where 
R equals a scale factor in each dimension (x and y). For 
purposes of this explanation we will assume that the aspect 
ratios of the HD picture and the SD picture are the same and 
that R=2. One SD macroblock (mbl) therefore corresponds 
to four HD macroblocks (MBl, MB2, MB3 and MB4). 

[0035] If the SD picture to be formed from the transcoding 
process is to have an aspect ratio which is diflfercnl, for 
example 4 by 3, side panels may be used to select the HD 
video area so that the samee aspect ratio can be preserved. 
For each macroblock in SD, one can map it to the corre- 
sponding area in HD. All the HD macroblocks fully or 
partially covered by this area are used as the co-«ited 
macroblocks of that particular SD macroblock. The HD 
decoder 10 also provides: 

[0036] a)the number of bits used to code each respec- 
tive macroblock, to buffer control and adaptive quan- 
tizer 30; 

[0037] b) quantizer information (qHD) for each mac- 
roblock, to buffer control and adaptive quantizer 30; 
and 

[0038] c) motion vector information for each mac- 
roblock, to motion estimation processor 60. 

[0039] In addition to the above information about indi- 
vidual macroblocks, the decoder 10 provides information 
about the type of picture each HD macroblock is part of (for 
example I, P or B) to modules 20,30,50 and 60 of the 
transcoder. 
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TABLE I 



Priority List of Macroblock TVpes (Top: High, Bottom: Low) 

Picture Type I P B 

Intja Intra latra 

Field predicted Field predicted 
Frame predicted Frame predicted 

Forward if only forward 
Backward if only back- 
ward 

Bi-directional if both exist 



[0040] Each group of four 16x16 pixel HD macroblocks is 
used to derive a co-sited 16x16 pixel SD macroblock. 

[0041] The macroblock type of each SD macroblock is 
determined in mode selection processor 50 based upon mode 
information from co-sited HD macroblocks. 

[0042] The macroblock mode relationship can be repre- 
sented in the following general terms: 

r(mfcl)=Ctn (Affll), Tl (Mm), 73 (MBZ) . . . ,Tn 

[0043] where G is a function or operatioDj n is the total 
number of the co-sited HD macroblocks, t is the mode to be 
assigned to SD macroblock mbl, and Tl to Tn are the 
respective modes of the co-sited macroblocks. 

[0044] The SD macroblock mode can be determined, for 
example, by a process G which determines the respective 
macroblock modes of each HD macroblocks in the group of 
HD macroblocks MBl to MBn and keeps count of the 
number of times each particular mode is xised in order to 
determine which mode is used most often in the group. The 
mode used most often in the group of HD macroblocks is 
then assigned to SD macroblock mbl. Likewise, the type of 
DCT coding most often used to code the residual data among 
each of the co-sited macroblocks MBl to MBn, is iised to 
determine the type of DCT coding to be used in SD 
macroblock mbl. 

[0045] In case there is no mode which represents a plu- 
rality (i.e. in case of a "tie"), the priority list in Table I can 
be applied lo determine prediction type for a particular SD 
macroblock. 

[0046] Table 1 is based on the heuristics to maximize the 
overall coding performance for the SD video. 

[0047] For each HD picture, the corresponding SD picture 
will have the same picture type (1, P or B). Table I is 
organized according to the picture type. The possible mac- 
roblock categories which can be assigned to the SD mac- 
roblock are listed with the highest priority at the top and the 
lowest priority at the bottom of the column. The use of Table 
I can be illustrated by the following example. 

[0048] If the HD co -sited macroblocks are part of a P 
picture, then the corresponding SD macroblock will be 
determined for a P picture (column with the heading P). If 
there is a "lie" between two intra coded co -sited macrob- 
locks and two field predicted macroblocks, the SD macrob- 
lock would be intra coded since of the two categories of HD 
macroblock, intra is the highest. 

[0049] For each picture type, DCT coding type (i.e. frame 
or field based) is also determined by plurality. In case of a 
"tie", field DCT is selected for the SD macroblock. 



[0050] After the SD macroblock mode is selected the 
motion vectors of the SD macroblock can be determined. 
Motion compensation is always performed on each SD 
macroblock in SD encoder 40 based on its SD macroblock 
mode and its motion vectors. Along with the respective 
derived SD macroblock type, the motion vectors for each 
group of HD macroblocks are used to determine the motion 
vectors for a co^ited SD macroblock. 

[0051] For an intra SD macroblock, no motion vectors are 
used. 

[0052] For a forward predicted SD macroblock the for- 
ward frame or field motion vector of mbl is a function of the 
forward frame and field motion vectors for the group of HD 
macroblocks MBl to MB4, as shovra in FIG. 3 and 
explained in more detail below. 

[0053] For a bidirectional predicted SD macroblock, the 
forward frame or field motion vector of mbl is a function of 
the forward frame and field motion vectors for MBl to MB4. 
The backward frame or field motion vector of mbl is a 
function of the backward frame and field motion vectors for 
MBl to MB4. Once the initial estimates of the motion 
vectors for mbl have been determined, additional motion 
estimation with these motion vectors ofiEset may be carried 
out for further refinement. 

[0054] The motion compensation is then performed by the 
SD encoder 40 to re -calculate the residues. 

[0055] Initial motion vectors for SD macroblock mbl are 
estimated in motion estimation processor 60 using the HD 
motion vectors supplied by HD decoder 10 by the methods 
described below. 

[0056] The initial estimate of the motion vector for mbl 
can be determined, for example, by dividing the average of 
motion vectors of HD macroblocks MBl to MBn in each 
direction (forward/backward) by R. In otherwords from 
MBl to MBn, the motion vectors belonging to the same 
direction (forward/backward) regardless of their structures 
(field/frame) should be averaged. 

[0057] If the required estimate of the initial motion vector 
is frame based, then all the HD field motion vectors are 
converted to the corresponding HD frame motion vectors, in 
the motion estimation processor 60, by dividing the vertical 
component of the respective field motion vector by two 
before averaging. If the required estimate of the initial 
motion vector is field based, then all the frame motion 
vectors are converted to the corresponding field motion 
vectors before averaging by multiplying the vertical com- 
ponent of these frame motion vectors by two. 

[0058] Since it is very likely that the co-sited HD mac- 
roblocks in each group have different types and values of 
motion vectors, the initial motion vectors for the co-sited SD 
macroblock derived directly from these HD motion vectors 
may not be very accurate. Additional motion estimation as 
the refinement is therefore required. Given a good initial 
estimate from the co-sited HD motion vectors, the amount of 
motion estimation needed is still much less than that 
required by a complete SD encoder. 

[0059] In order to allow constant rate transmission of 
bursty (compressed) data, buffers are needed at the encoder 
and the decoder. Consider the broadcast scenario, where 
there will be one encoder buffer and innumerable decoder 
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buffers (with information transmitted only in one direction: 
from the encoder to the decoder). The encoder must ensure 
that none of the decoder buffers either overflow or under- 
flow. MPEG addresses this problem by having the encoder 
generate a video buffer status signal (vbv_delay). The 
encoder transmits the vbv_delay for every picture (also 
referred to as a frame) in order to inform the decoder of the 
state in which its buffer should be before the start of 
decoding of the current picture. 

[0060] Since the transcoder generates an SD bitstream 
from an HD bitstream, it needs to make sure that the SD 
bitstream satisfies the constraints imposed by the need for 
video buffer control in the SD decoder. In other words, since 
the bit-rate and buffer size change for the SD bitstream, the 
video buffer control information of the HD bitstream (HD 
vbv_delay) will have to be modified to appropriate video 
buffer control information for the transcoded SD bitstream 
(SD vbv_delay). This is achieved in buffer control and 
adaptive quantizer 30 in the following manner. 

[0061] The HD encoder ensures that the video buffer 
conditions are satisfied by providing an HD vbv-delay 
signal, as taught in the references incorporated herein. 
Mathematically, the requirement of the HD video buffer 
control information is: 

0<OHD<OHD 

[0062] where, OHD is the occupancy of a video buffer of 
a hypothetical decoder coupled to an HD encoder, immedi- 
ately before and immediately after decoding a frame, and, 
BHD is the size of the buffer. 

[0063] The corresponding requirement of the SD video 
buffer control information is, simflarly: 

0<OSD<BSD 

[0064] where, OSD is the occupancy of a video buffer of 
a hypothetical decoder coupled to an SD encoder, immedi- 
ately before and immediately after decoding a frame, and, 
BSD is the size of the buffer. This requirement can be 
satisfied by using buffer control and adaptive quantizer 30 in 
the transcoder which comprises a buffer controller which 
takes advantage of the fact that if the relationship between 
buffer occupancy and buffer size stated above is true, a 
restriction on OSD is imposed as defined as follows: 

OSDm{BSD{BHD)OHD. 

[0065] The buffer control and adaptive quantizer 30 
receives the number of bits used for encoding each HD 
macroblock and computes the number of bits used to encode 
each HD picture using the formula: 

BnS_SDi-{BSD(BHD)BrrS^Di 

[0066] where, BITS_SDi is the estimated number of bits 
to be used to code a corresponding SD picture i and 
BlTS_HDi is the actual number of bits used to code the 
corresponding HD picture i. 

[0067] It should be noted that this requires that the control 
over the buffer occupancy be tight. la other words, the actual 
coded bits per picture must be close to the estimated number 
of target bits. 

[0068] After BITS_SDi is computed for the current SD 
picture, an average quantizer scale (Qiso) for 'he SD picture 
which would result in BITS_SDi is computed in buffer 
control and adaptive quantizer 30 as follows: 



[0069] The complexity of the SD picture (Ci) is repre- 
sented by the formula 

Ci'BrrS_SDi*Qi^^ 

[0070] and similarly, for the previous SD picture, 

Ci-UBITS_$Di-\ •QMsd- 

[0071] In order to achieve a continuity in quality from 
picture to picture, Ci should be equal to Ci-1. Therefore, 
solving for estimated average quantizer step size, we get; 

QU^''{BITS_SDi-VQi-^MSn'S_SDi') 

[0072] The quantizer step size value for each of the HD 
macroblocks of the co-sited HD picture (qHD) are provided 
by HD decoder 10 to buffer control and adaptive quantizer 
30. Buffer control and adaptive quantizer 30 calculates the 
average value of the qHo values for the HD picture in order 
to provide an average quantizer scale, Qj^j. for the current 
HD picture. Buffer control and adaptive quantizer 30 can 
also calculate QiSD since Qi-lso BITS_SDi-l are 
retained in its buffer. At the beginning of the transcoding 
process, before a first value of Qi-lgo available, QHD can 
be used in place of Qi-lsp. 

[0073] The bits available to code an SD picture must be 
allocated in such a manner as to increase its subjective 
quality without increasing the number of bits used to encode 
it. This is achieved by allowing more distortion in those 
areas of the image which are complex (where it is less 
visible) while quantizing finely those areas that are sensitive 
to noise (for instance, flat and low activity areas). 

[0074] The quantization step size ^^o for ^^^^ rnac- 
roblock, which determines the distortion introduced in the 
macroblock and determines the number of bits generated by 
it, is computed by buffer control and adaptive quantizer 30 
from three factors: (1) the estimated average quantizer step 
size, QsD, (2) the buffer status, OSisD» (which is obtained by 
reading the current occupancy of the buffer in buffer control 
and adaptive quantizer 30), and (3) the relative complexity 
of each SD macroblock with respect to the other macrob- 
locks of its SD picture. As described in the following 
equation: 

q^v>'{Qi, OSDi, cj) 

[0075] where the average complexity of a macroblock, cj, 
is a function of the following quantities obtained from the 
HD coded bitstream: 

cj-f{qHDl»qHD2,qHD3.qHD4 • • • qnHD'. bl,b2,b3,b4 . . 
. bn) 

[0076] where b is nimiber of bits for each HD macroblock. 
For example: 

[0077] cjominimum value from among the foUowing 
products: (blqHol); (b2q„D2); (b3q„D3); (b4qHD4). 
The rationale for the above procedure is that coding 
the SD macroblock based on the HD-macroblock 
that is most sensitive to noise ensures that the 
"worst" case is taken care of. For the same quanti- 
zation step, fewer number of bits means lower activ- 
ity thus most sensitive to noise. 

[0078] While the invention has been described in its 
preferred embodiment, it is to be understood that the words 
which have been used are words of description rather than 
of limitation and that changes within the purview of the 
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appended claims may be made without departing from the 
true scope and spirit of the invention in its broader aspects. 

We claim: 

1. A method for transcoding a first group of macroblocks 
of a first digital television signal, to a co -sited macroblock 
of second digital television signal, said method comprising 
the steps of: 

a) deriving from each of said first group of macroblocks, 
corresponding HD macroblock information; and 

b) deriving SD macroblock information for said co-sited 
macroblock directly from said HD macroblock infor- 
mation so as to form said second digital television 
signal. 

2, The method of claim 1 wherein said HD macroblock 
information and said SD macroblock information comprises 
prediction information, quantizer scale information, motion 
information and transform coefficient information. 



3. An apparatus for transcoding a first group of macrob- 
locks of a first digital television signal, to a co-sited mac- 
roblock of second digital television signal, said apparatus 
comprising: 

a) means for deriving from each of said first group of 
macroblocks, corresponding HD macroblock informa- 
tion; and 

b) means for deriving SD macroblock information for said 
co-sited macroblock directly from said HD macroblock 
information so as to form said second digital television 
signal, 

4. The apparatus of claim 3 wherein said HD macroblock 
information and said SD macroblock information comprises 
prediction information, quantizer scale information, motion 
information and transform coefficient information. 

♦ ♦ * ♦ ♦ 
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